Notes on Hidden Markov Models

The model A hidden Markov model is characterized by a set of M states, by an initial probability distribution for the first state, by a transition probability matrix linking successive states, and by a state-dependent probability distribution on the outputs. We represent the state at time t as a multinomial random variable qt, with components q t, for i = 1, . . . ,M . Thus q t is equal to one for a particular value of i and is equal to zero for j 6= i. We use a subscript to denote the time step, thus qt is the multinomial state at time t. The transition probability matrix A is the probability of transitioning between the multinomial states at successive time steps; in particular, the (i, j)th entry aij is the transition probability P (q t+1 = 1|q t = 1). Note that we assume that this transition probability is constant as a function of t; that is, we assume a homogeneous hidden Markov model. All of the algorithms that we describe are readily generalized to the case of a varying A matrix, however this case is less common in practice than the homogeneous case. We also need an initial condition. The vector π represents the probability distribution on the initial state; in particular, we have πi = P (q 1 = 1). There are three related graphical representations of hidden Markov models that it is important to distinguish. The first representation, shown in Figure 1, is the stochastic automaton. In this diagram, the components of the multinomial state are shown as separate nodes and the arcs represent the transition probabilities. This diagram is not a graphical model; in particular there are cycles in the graph and the arcs do not represent assertions of conditional independence. The